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METHOD AND DEVICE FOR RECOGNIZING 
VISUAL OBSTRUCTIONS IN IMAGE SENSOR SYSTEMS 



Background Information 
FIELD OF THE INVENTION 

The present invention relates to a method and to a device for 
recognizing visual obstructions in image sensor systems. 

5 

BACKGROUND INFORMATION 

In the future, in particular in conjunction with environmental 
sensing of motor vehicles, image sensors such as video 
cameras, for example, will may be used whose image is 

10 evaluated by downstream functions (driver assistance 

functions, for example) . It is therefore of particular 
importance to recognize visual obstructions which degrade the 
image detected by the image sensor, to inform the downstream 
functions or systems of the presence of such visual 

15 obstructions, and to initiate countermeasures^ if necessary. 

Advantages of — the — Invention 
SUMMARY OF THE INVENTION 

The detection of the blurriness of the recorded image results 
20 in a suitable procedure method which assists in recognizing 
visual obstructions, ^-fc — is pax ' ticularly advantageous — that 
downstream Downstream systems or functions, which function 
properly only when the view is adequate, are may be informed 
of the presence of the visual obstruction and are thus able to 
25 initiate suitable countermeasures . When a visual obstruction 
is recognized, depending on the design configuration of the 
downstream function or systemj_ the recorded image is 
advantageously not evaluated or corrected, or other suitable 
measures, such as switching on the windshield wiper system or 
30 windshield heating, for example, are performed. 
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— is parfcic.ular ly advantageous that visual Visual 
obstructions are may be recognized based on the image signal 
itself, so that no additional sensors are necessary. 

5 The procedure method is suitable in particular for recognizing 
a visual obstruction on the windshield of the motor vehicle. 

The image sensor advantageously takes over assumes additional 
functions, such as the function of a rain sensor, for example, 
10 so that this additional rain sensor may be dispensed with in 
the furnishing of — the vehicle with such image sensor-based 
driver assistance functions, for example lane monitoring 
systems . 

15 The procedure method described below -ars may be used in a 

particularly advantageous mann e r in video sensor systems in 
motor vehicles^, which systems are not focused on the vehicle 
windshield, but rather on the external region. The procedure 
method described below may thus be used to particular 

2 0 advantage in conjunction with the sensing of the environment 

around a motor vehicle. In this manner the image sensor may be 
used for multiple applications (for example, rain sensor and 
object detection, etc.). 

25 — is also advantageous that — the The image sensor d-s may be 
able to check its functionality using its own output signal, 
i.e., to determine whether the instantaneous view conditions 
are sufficient for the next function that is to be performed. 
When the image sensor is not functioning properly, this is 

3 0 advantageously signaled to the driver or the downstream 

system, and/or countermeasures are taken. 

It has been shown in a particularly advantageous manner that 
the The detection of visual obstructions « may be performed 
35 by measuring the blurriness of the represented image. A 

reliable method is thus may be provided for determining visual 
obstructions, in particular objects on a vehicle windshield, 
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which appear blurry in the image. It is particulairly 
advantageous that — transparent Transparent objects such as 
raindrops, or semi transparent objects such as ice or dust, for 
example, mre may be recognized, and in one particular 
exemplary embodiment even distinguished from one another, by 
measuring the blurriness of the image. 

Thus, -rt — is also advantageously possible to obtain information 
may be obtained about the type of visual obstructions. In this 
manner^ a response specific to the particular type of visual 
obstruction may be made, for example by automatically 
switching on the windshield washer system when the windshield 
is soiled by particles. 

Further advantages arise from the following desciription of 
exemplary embodiments and the de p endent c laims. 

Drawing 

The present — invention is explained in greater detail below 
with reference to the exemplary embodiments illustrated in the 
drawing . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows an overview a schematic illustration of an 
image sensor system im by which the procedure a method 
described below according to the present invention for 
determining visual obstructions is implemented-; — and _ ^ 

Figures 2 through 4 show diagrams with reference to which 
three different — embodiment s 

Figures 2a and 2b show charts illustrating an exemplary 
embodiment of the method for determining the blurriness in the 
image and the recognition of visual obstructions derived 
therefrom are described . 
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Figures- 3a and 3b show charts illustrating another exemplary 
embodiment of the method for determining the blurriness in the 



image and the recognition of visual obstructions derived 
therefrom. 

5 

Figures 4a and 4b show charts illustrating another exemplary 
embodiment of t h e method for de t erminincf the blurriness in the 
image and the recognition of visual obstructions derived 
therefrom. 

10 

Description of — the Exemplax - y — Embodiment s 
DETAILED DESCRIPTION 

The image sensor system illustrated in Figure 1 is composed 

15 essentially of includes an optical sensor array 3. which is 

connected to a data processing unit (or an evaluation unit) 7, 
for example a microprocessor. The instantaneously visible 
scene is mapped onto the sensor array 3. by a lens system 2. 
The image sensor is mounted behind a windshield 1 and is 

2 0 focused on the an external region behind beyond the 

windshield. The windshield is, for example, the window glass 
of a vehicle, in particular the windshield of the vehicle. The 
focusing on the vehicle's external region allows the image 
sensor system to also be used for other image processing 

25 operations such as lane detection or obstruction detection, 
for example. Evaluation unit 7 is composed of includes a 
module 4 for controlling lighting of the sensor array, a 
measuring unit 5 for estimating blurriness, and a decision 
module 6 which makes decisions regarding the presence and, 

30 optionally, the type, of a visual obstruction. In one 

preferred exemplary embodiment all of these modules are part 
of a microprocessor, the modules being implemented as software 
programs. In other exemplary embodiments, the modules are 
individual components (processors) , likewise implemented as 

35 software, where estimations of blurriness are made, for 

example, in one component and decisions are made in another 
component. The decision made by decision module 6 is then 
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relayed, as an information and/or activation signal to 
downstream functions and/or systems. Application possibiliti e s 
are described in greater detail below. 



5 The automatic detection of visual obstructions on the 

windshields of a vehicle therefore plays an increasingly 
important role due to the fact that, when image sensors are 
used in the vehicle, systems which evaluate the images from 
the sensor function properly only when the view is adequate. 

10 If there is information about the visual obstruction and, in 
the ideal case, about the type of visual obstruction, such a 
system is able to display to the driver its instantaneous lack 
of functionality and/or to take count ermeasures , such as 
switching on the wiper system, the windshield heating system , 

15 or the windshield washer system, for example. 

When visual obstructions are determined based solely on the 
image sensor signal, the problem arises that visual 
obstructions on a vehicle windshield are made noticeable in 

20 the image only indirectly, since -for — crse in driver assistance 
systems the image sensor is focused on the vehicle's external 
region. Objects on the windshield, for example raindrops or 
dust, are therefore blurrily imaged. These visual obstructions 
are thus made noticeable by a characteristic distribution of 

25 the blurriness in the image signal. 

The detection of visual obstructions is based on measuring the 
blurriness of the image which is recorded by an image sensor. 
This blurriness is made noticeable by a lack of sharpness of 

30 the imaged contours of the instantaneously visible scene. 

Objects present on the windshield are blurrily imaged due to 
the focusing of the image sensor on the external region. 
Transparent objects, such as raindrops, for example, result in 
localized defocusing of the image, and semitransparent objects 

35 such as ice or dust scatter incident light beams. Both effects 
result in an increase in the blurriness of the sensor signal . 
The evaluation unit which is associated with the image sensor, 
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an optical sensor array of, for example, a CCD or CMOS camera, 
detects the distribution of blurriness of the instantaneously 
imaged scene. Based on the detected distribution of 
blurriness, a conclusion is drawn as to the presence of a 
visual obstruction. One of — bite The methods for digital image 
processing described below irs may be used for determining the 
blurriness . 

In one pref erred exemplary embodiment, the image sensor is a 
sensor array which assists in obtaining information also on 
the type of visual obstruction. This fundamentally 
distinguishes such a sensor array from a conventional rain 
sensor which operates by the reflection principle. The use of 
a sensor array allows a specific response to be initiated in 
response to the determined visual obstruction by, for example,, 
by additionally switching on a windshield washer system for 
washincf soiling caused by particles. One preferred p irocedur e 
exemplary method for recognizing the type of visual 
obstruction is likewise described in detail below. 

A first possibility for measuring blurriness is illustrated 
with reference to the diagrams in Figure 2 Figures 2a and 2b . 
Blurriness is measured here using the contrast spectrum. The 
underlying principle of this method is known from the field of 
digital image processing. To obtain the contact spectrum, 
first a multiscale analysis is carried out performed, in which 
the video image is decomposed into multiple images of 
decreasing resolution by the repeated use of a smoothing 
operation and subsequent subsampling. A global measure of 
contrast, for example the standard deviation of the intensity 
values in the image, is calculated in each resolution stage. 
The measure of contrast plotted as a function of the 
resolution forms the contrast spectrum of the image. Examples 
of such contrast spectra are shown in Figures 2a and 2b. In 
these figures, the particular measure of contrast K of the 
image is plotted as a function of resolution A. Sharp and 
blurry images differ in their contrast spectra by the fact 
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that the drop in contrast as a function of increasing 
resolution is steeper in blurry images than in sharp images. 
This is because fine details are degraded more strongly by 
visual obstructions than are coarse image features. Thus, the 
drop in the contrast spectrum is a measure of the blurriness 
of the image . Situations are shown as exam p les in Figures 2a 
and 2b. Figure 2a illustrates a contrast spectrum as a 
function of the image resolution for an image with slight 
blurriness, while Figure 2b shows the situation for high image 
blurriness (blurry image) . 

In one implementation of the procedure method for determining 
the blurriness of the image, the contrast spectrum is recorded 
by multiscale analysis within a computing program, and a 
variable is determined, for example the average increase in 
the measure of contrast as a function of the resolution, which 
characterizes the variation of the measure of contrast as a 
function of the resolution. The blurriness of the image is 
determined by comparing this variable to at least one limiting 
value. For recognizing a visual obstruction, in particular 
with regard to the response measures planned in the downstream 
systems, it is sufficient to determine that the image is 
blurry [[;]] ^ thus Thus, the determined variable is compared to 
a limiting value, and if this limiting value is exceeded, 
blurriness may be assumed, or the variable itself is relayed 
as a measure of blurriness. In other applications, determining 
a measure of the blurriness « may be desired. In these cases, 
limiting values form ranges for the variable which 
characterizes the contrast spectrum and to which blurriness 
values are assigned. A value for the magnitude of the 
blurriness is determined when the value characterizing the 
contrast spectrum is in a specified range of values. 

In addition to measuring the blurriness by using the contrast 
spectrum, there are a number of other methods from the field 
of digital image processing that may be used to detect 
blurriness. Examples of such alternative methods for measuring 
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the image blurriness are measures for analyzing the Fourier 
spectrum or the autocorrelation of the analyzed image. In the 
Fourier spectrum a blurry image is characterized by the fact 
that the amplitudes of the high spatial frequencies, which 
represent the finer image details, are strongly attenuated in 
comparison to a sharp image. 

Examples of such a situation are shown in Figures 3a and 3b. 
The curve of the Fourier spectrum is plotted in both figures, 
the amplitude of the Fourier transformation function being 
represented by the spatial frequency. Figure 3a shows a 
slightly blurry image, while Figure 3b shows a highly blurry 
image. It is obvious that the amplitudes at the higher spatial 
frequency decrease greatly for a blurry image. For evaluation, 
±rt one preferred exemplary embodiment -irt — i-s — therefore provided 
of the present invention provides that the amplitude and the 
corresponding spatial frequency are used to recognize values 
that fall below a threshold value. If this spatial frequency 
is below a predetermined value, it is assumed that the image 
is blurry. In another exemplary embodiment, the slope of the 
curve could also be used here to determine blurriness by 
comparing it to at least one limiting value. For the 
evaluation of the Fourier spectrum as well, -tb — is possible to 
determine a value for the magnitude of blurriness may be 
determined by specifying multiple limiting values or by 
specifying value ranges, and relaying this value to downstream 
systems . 

A third possibility for measuring blurriness is the 
autocorrelation function of the image. This autocorrelation 
function decreases less steeply for a blurry image than for a 
sharp image as a function of distance. This is because only 
large structures remain in a blurry image. Figure 4 
illustrates Figures 4a and 4b illustrate this relationship, 
the particular magnitude of the autocorrelation function being 
plotted as a function of the pixel separation. Figure 4a shows 
the autocorrelation function for a slightly blurry image, and 
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Figure .4b shows the autocorrelation function for a highly 
blurry image. The figures show that as the pixel spacing for 
an image having a low degree of blurriness (sharp image) 
increases, the autocorrelation function rapidly decreases, 
whereas for a blurry image the decrease in the autocorrelation 
function is less pronounced , As indicated above, for the 
evaluation it is possible to use at least one limiting value 
or multiple limiting values being exceeded or not being 
attained, or, in another exemplary embodiment, to use the 
calculated average increase to determine the blurriness and/or 
to determine a value for the magnitude of the blurriness. 

A further alternative for measuring blurriness -irs may be 
provided by an indirect classification formulation in which a 
learning machine, a neural network or a polynomial classifier*, 
for example, is trained to distinguish between blurry and 
sharp images by presenting a large number of images as 
examples . 

As mentioned above, a visual obstruction is assumed when a 
high degree of blurriness is recognized, or the value of the 
blurriness is greater than a predetermined value. In addition, 
a comparison to reference distributions for contrasts, Fourier 
components, or autocorrelations may differentiate various 
types of visual obstructions based on their similarity to 
specific reference distributions. In this manner -irt — irs 
possible to distinguish^ moisture on the windshield may be 
distinguished from ice or dust on the windshield, and, 
depending on the situation, to initiate different 
countermeasures, for example, activation of the windshield 
wiper, switching on a windshield heating system or a 
windshield washer system, etc. Here as well, the 
above -referenced classification formulation may be provided by 
learning machines. 

If the video sensor having visual obstruction detection is 
also operated as a rain sensor, further additions are useful 
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applications, are possible . If moisture on the windshield is 
recognized based on the determined blurriness and the contrast 
distribution, or the Fourier spectrum or autocorrelation, the 
windshield wiper is, as previously, initially actuated only at 
5 the explicit request of the driver, to prevent erroneous 

start-up when an error is detected. Using the video images 
which are recorded immediately after the wiping operation, ±t: 
is possible to obtain the above-mentioned reference 
distributions may be obtained , using which a decision is made 

10 to initiate the next wiping operation. Thus, the wiping 

response may be adapted on a situation-specific basis. For 
situations in which the surrounding scene contains too little 
contrast, it may be practical to temporarily switch on a 
windshield light. Visual obstructions are then detectable from 

15 the scattering of the light. 
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Abstract 

ABSTRACT OF THE DISCLOSURE 
A method and a device for recognizing visual obstructions in 
image sensor systems are proposed, provided . sc A signal being 
5 ±s produced, based on the analysis of the recorded image from 
the image sensor, which indicates the presence and, 
optionally, the type, of the visual obstruction on a 
windshield, for example . 

10 (Figure 1) 
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METHOD AND DEVICE FOR RECOGNIZING 
VISUAL OBSTRUCTIONS IN IMAGE SENSOR SYSTEMS 

FIELD OF THE INVENTION 

The present invention relates to a method and a device for 
recognizing visual obstructions in image sensor systems. 

5 BACKGROUND INFORMATION 

In the future, in particular in conjunction with environmental 
sensing of motor vehicles, image sensors such as video 
cameras, for example, may be used whose image is evaluated by 
downstream functions (driver assistance functions, for 
10 example) . It is therefore of particular importance to 

recognize visual obstructions which degrade the image detected 
by the image sensor, to inform the downstream functions or 
systems of the presence of such visual obstructions, and to 
initiate countermeasures, if necessary. 

15 

SUMMARY OF THE INVENTION 

The detection of the blurriness of the recorded image results 
in a suitable method which assists in recognizing visual 
obstructions. Downstream systems or functions, which function 

20 properly only when the view is adequate, may be informed of 
the presence of the visual obstruction and are thus able to 
initiate suitable count ermeasure s . When a visual obstruction 
is recognized, depending on the configuration of the 
downstream function or system, the recorded image is not 

25 evaluated or corrected, or other suitable measures, such as 
switching on the windshield wiper system or windshield 
heating, for example, are performed. 

Visual obstructions may be recognized based on the image 
30 signal itself, so that no additional sensors are necessary. 
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The met-hod i-s suitable in particular for recognizing a visual 
obstruction on the windshield of the motor vehicle. 

The image sensor assumes additional functions, such as the 
5 function of a rain sensor, for example, so that this 

additional rain sensor may be dispensed with in the vehicle 
with such image sensor-based driver assistance functions, for 
example lane monitoring systems. 

10 The method described below may be used in video sensor systems 
in motor vehicles, which systems are not focused on the 
vehicle windshield, but rather on the external region. The 
method described below may thus be used in conjunction with 
the sensing of the environment around a motor vehicle. In this 

15 manner the image sensor may be used for multiple applications 
(for example, rain sensor and object detection, etc.). 

The image sensor may be able to check its functionality using 
its own output signal, i.e., determine whether the 
20 instantaneous view conditions are sufficient for the next 

function that is to be performed. When the image sensor is not 
functioning properly, this is signaled to the driver or the 
downstream system, and/or countermeasures are taken. 

25 The detection of visual obstructions may be performed by 

measuring the blurriness of the represented image. A reliable 
method may be provided for determining visual obstructions, in 
particular objects on a vehicle windshield, which appear 
blurry in the image. Transparent objects such as raindrops, or 

3 0 semi transparent objects such as ice or dust, for example, may 
be recognized, and in one exemplary embodiment even 
distinguished from one another, by measuring the blurriness of 
the image . 

3 5 Thus, information may be obtained about the type of visual 
obstructions. In this manner, a response specific to the 
particular type of visual obstruction may be made, for example 
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by automatically switching on the windshield washer system 
when the windshield is soiled by particles. 

BRIEF DESCRIPTION OF THE DRAWINGS 
5 Figure 1 shows a schematic illustration of an image sensor 
system by which a method according to the present invention 
for determining visual obstructions is implemented. 

Figures 2a and 2b show charts illustrating an exemplary 
10 embodiment of the method for determining the blurriness in the 
image and the recognition of visual obstructions derived 
therefrom. 

Figures 3a and 3b show charts illustrating another exemplary 
15 embodiment of the method for determining the blurriness in the 
image and the recognition of visual obstructions derived 
therefrom. 

Figures 4a and 4b show charts illustrating another exemplary 

2 0 embodiment of the method for determining the blurriness in the 

image and the recognition of visual obstructions derived 
therefrom. 

DETAILED DESCRIPTION 
25 The image sensor system illustrated in Figure 1 includes an 

optical sensor array 3 which is connected to a data processing 
unit (or an evaluation unit) 1 , for example a microprocessor. 
The instantaneously visible scene is mapped onto the sensor 
array 3 by a lens system 2 . The image sensor is mounted behind 

3 0 a windshield 1 and is focused on an external region beyond the 

windshield. The windshield is, for example, the window glass 
of a vehicle, in particular the windshield of the vehicle. The 
focusing on the vehicle's external region allows the image 
sensor system to also be used for other image processing 
35 operations such as lane detection or obstruction detection, 
for example. Evaluation unit 7 includes a module 4 for 
controlling lighting of the sensor array, a measuring unit 5 
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for estimating blurriness, and a decision module 6 which makes 
decisions regarding the presence and, optionally, the type, of 
a visual obstruction. In one exemplary embodiment all of these 
modules are part of a microprocessor, the modules being 
5 implemented as software programs. In other exemplary 
embodiments, the modules are individual components 
(processors) , implemented as software, where estimations of 
blurriness are made, for example, in one component and 
decisions are made in another component. The decision made by 
10 decision module 6 is then relayed as an information and/or 
activation signal to downstream functions and/or systems. 

The automatic detection of visual obstructions on the 
windshields of a vehicle plays an important role due to the 

15 fact that, when image sensors are used in the vehicle, systems 
which evaluate the images from the sensor function properly 
only when the view is adequate. If there is information about 
the visual obstruction and, in the ideal case, about the type 
of visual obstruction, such a system is able to display to the 

20 driver its instantaneous lack of functionality and/or to take 
countermeasures, such as switching on the wiper system, the 
windshield heating system, or the windshield washer system, 
for example. 

25 When visual obstructions are determined based solely on the 
image sensor signal, the problem arises that visual 
obstructions on a vehicle windshield are made noticeable in 
the image only indirectly, since in driver assistance systems 
the image sensor is focused on the vehicle's external region. 

3 0 Objects on the windshield, for example raindrops or dust, are 
therefore blurrily imaged. These visual obstructions are thus 
made noticeable by a characteristic distribution of the 
blurriness in the image signal , 

35 The detection of visual obstructions is based on measuring the 
blurriness of the image which is recorded by an image sensor. 
This blurriness is made noticeable by a lack of sharpness of 
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the imaged contours of the instantaneously visible scene. 
Objects present on the windshield are blurrily imaged due to 
the focusing of the image sensor on the external region. 
Transparent objects, such as raindrops, for example, result in 
5 localized defocusing of the image, and semitransparent objects 
such as ice or dust scatter incident light beams. Both effects 
result in an increase in the blurriness of the sensor signal . 
The evaluation unit which is associated with the image sensor, 
an optical sensor array of, for example, a CCD or CMOS camera, 
10 detects the distribution of blurriness of the instantaneously 
imaged scene. Based on the detected distribution of 
blurriness, a conclusion is drawn as to the presence of a 
visual obstruction. The methods for digital image processing 
described below may be used for determining the blurriness . 

15 

In one exemplary embodiment, the image sensor is a sensor 
array which assists in obtaining information also on the type 
of visual obstruction- This fundamentally distinguishes such a 
sensor array from a conventional rain sensor which operates by 

20 the reflection principle. The use of a sensor array allows a 
specific response to be initiated in response to the 
determined visual obstruction by, for example, by additionally 
switching on a windshield washer system for washing soiling 
caused by particles. One exemplary method for recognizing the 

25 type of visual obstruction is described in detail below. 

A first possibility for measuring blurriness is illustrated 
with reference to the diagrams in Figures 2a and 2b. 
Blurriness is measured here using the contrast spectrum. The 

30 underlying principle of this method is known from the field of 
digital image processing. To obtain the contact spectrum, 
first a multiscale analysis is performed, in which the video 
image is decomposed into multiple images of decreasing 
resolution by the repeated use of a smoothing operation and 

35 subsequent subsampling. A global measure of contrast, for 

example the standard deviation of the intensity values in the 
image, is calculated in each resolution stage. The measure of 
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contrast plotted as a function of the resolution forms the 
contrast spectrum of the image. Examples of such contrast 
spectra are shown in Figures 2a and 2b. In these figures, the 
particular measure of contrast K of the image is plotted as a 
5 function of resolution A. Sharp and blurry images differ in 
their contrast spectra by the fact that the drop in contrast 
as a function of increasing resolution is steeper in blurry 
images than in sharp images. This is because fine details are 
degraded more strongly by visual obstructions than are coarse 
10 image features. Thus, the drop in the contrast spectrum is a 
measure of the blurriness of the image. Figure 2a illustrates 
a contrast spectrum as a function of the image resolution for 
an image with slight blurriness, while Figure 2b shows the 
situation for high image blurriness (blurry image) . 

15 

In one implementation of the method for determining the 
blurriness of the image, the contrast spectrum is recorded by 
multiscale analysis within a computing program, and a variable 
is determined, for example the average increase in the measure 

20 of contrast as a function of the resolution, which 

characterizes the variation of the measure of contrast as a 
function of the resolution. The blurriness of the image is 
determined by comparing this variable to at least one limiting 
value. For recognizing a visual obstruction, in particular 

25 with regard to the response measures planned in the downstream 
systems, it is sufficient to determine that the image is 
blurry. Thus, the determined variable is compared to a 
limiting value, and if this limiting value is exceeded, 
blurriness may be assumed, or the variable itself is relayed 

30 as a measure of blurriness. In other applications, determining 
a measure of the blurriness may be desired. In these cases, 
limiting values form ranges for the variable which 
characterizes the contrast spectrum and to which blurriness 
values are assigned. A value for the magnitude of the 

35 blurriness is determined when the value characterizing the 
contrast spectrum is in a specified range of values. 
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In addition to measuring the blurriness by using the contrast 
spectrum, there are a number of other methods from the field 
of digital image processing that may be used to detect 
blurriness. Examples of such alternative methods for measuring 
5 the image blurriness are measures for analyzing the Fourier 

spectrum or the autocorrelation of the analyzed image. In the 
Fourier spectrum a blurry image is characterized by the fact 
that the amplitudes of the high spatial frequencies, which 
represent the finer image details, are strongly attenuated in 
10 comparison to a sharp image. 

Examples of such a situation are shown in Figures 3a and 3b. 
The curve of the Fourier spectrum is plotted in both figures, 
the amplitude of the Fourier transformation function being 

15 represented by the spatial frequency. Figure 3a shows a 

slightly blurry image, while Figure 3b shows a highly blurry 
image. It is obvious that the amplitudes at the higher spatial 
frequency decrease greatly for a blurry image. For evaluation, 
one exemplary embodiment of the present invention provides 

2 0 that the amplitude and the corresponding spatial frequency are 
used to recognize values that fall below a threshold value. If 
this spatial frequency is below a predetermined value, it is 
assumed that the image is blurry. In another exemplary 
embodiment, the slope of the curve could also be used here to 

25 determine blurriness by comparing it to at least one limiting 
value. For the evaluation of the Fourier spectrum as well, a 
value for the magnitude of blurriness may be determined by 
specifying multiple limiting values or by specifying value 
ranges, and relaying this value to downstream systems. 

30 

A third possibility for measuring blurriness is the 
autocorrelation function of the image. This autocorrelation 
function decreases less steeply for a blurry image than for a 
sharp image as a function of distance. This is because only 
35 large structures remain in a blurry image. Figures 4a and 4b 
illustrate this relationship, the particular magnitude of the 
autocorrelation function being plotted as a function of the 
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pixel separa-tion. Figure 4a shows the autocorrelation function 
for a slightly blurry image, and Figure 4b shows the 
autocorrelation function for a highly blurry image. The 
figures show that as the pixel spacing for an image having a 
5 low degree of blurriness (sharp image) increases, the 

autocorrelation function rapidly decreases, whereas for a 
blurry image the decrease in the autocorrelation function is 
less pronounced. As indicated above, for the evaluation it is 
possible to use at least one limiting value or multiple 
10 limiting values being exceeded or not being attained, or, in 
another exemplary embodiment, to use the calculated average 
increase to determine the blurriness and/or to determine a 
value for the magnitude of the blurriness . 

15 A further alternative for measuring blurriness may be provided 
by an indirect classification formulation in which a learning 
machine, a neural network or a polynomial classifier, for 
example, is trained to distinguish between blurry and sharp 
images by presenting a large number of images as examples. 

20 

As mentioned above, a visual obstruction is assumed when a 
high degree of blurriness is recognized, or the value of the 
blurriness is greater than a predetermined value. In addition, 
a comparison to reference distributions for contrasts, Fourier 

25 components, or autocorrelations may differentiate various 
types of visual obstructions based on their similarity to 
specific reference distributions. In this manner, moisture on 
the windshield may be distinguished from ice or dust on the 
windshield, and, depending on the situation, to initiate 

30 different countermeasures , for example, activation of the 

windshield wiper, switching on a windshield heating system or 
a windshield washer system, etc. Here as well, the above- 
referenced classification formulation may be provided by 
learning machines. 

35 

If the video sensor having visual obstruction detection is 
also operated as a rain sensor, further useful applications 
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are possible-. If moisture on the windshield is recognized 
based on the determined blurriness and the contrast 
distribution, or the Fourier spectrum or autocorrelation, the 
windshield wiper is, as previously, initially actuated only at 
the explicit request of the driver, to prevent erroneous 
start-up when an error is detected. Using the video images 
which are recorded immediately after the wiping operation, the 
above-mentioned reference distributions may be obtained, using 
which a decision is made to initiate the next wiping 
operation. Thus, the wiping response may be adapted on a 
situation-specific basis. For situations in which the 
surrounding scene contains too little contrast, it may be 
practical to temporarily switch on a windshield light. Visual 
obstructions are then detectable from the scattering of the 
light. 
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ABSTRACT OF THE DISCLOSURE 
A method and a device for recognizing visual obstructions in 
image sensor systems are provided. A signal is produced, 
based on the analysis of the recorded image from the image 
sensor, which indicates the presence and, optionally, the 
type, of the visual obstruction on a windshield, for example. 
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